Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Accelerating image processing for CNN #668

Merged
merged 8 commits into from
Dec 8, 2016

Conversation

qingqing01
Copy link
Contributor

@qingqing01 qingqing01 commented Nov 30, 2016

The image processing is a key step for CNN. It's slower when reading image list directly by PyDataProvider. And this leads to a lower speedup on Multi-GPUs, especially on 8 GPUs or 16 GPUs.

This PR has two features:

  1. A C++ accelerated library that can be called by Python: OpenCV + Multi-threads
  2. A Python module with python-multiprocessing and Python-OpenCV

One can select one of them to use.

The performance on 4 GPUs( K40 ) with input size 224 * 224 *3 is as follows.
实验条件:

  • 4 Tesla K40m
  • 总batch size: 192
  • 下面时间为20个mini-Batch的时间
  1. PIL直接读取图片list: 64.8s
  2. Pickle对数据打Batch再读取:46.8s
  3. 数据打Batch + Python多进程 + Python-OpenCV: 29.7s
  4. 数据打Batch + C++多线程 + OpenCV: 26.7s

下一个PR增加使用文档。

#457

Copy link
Collaborator

@reyoung reyoung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

有个基本问题,先把cpplint加到这些文件里

#include "paddle/utils/Thread.h"

using namespace cv;
using namespace paddle;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

不要在.h里面using namespace


#include "DataTransformer.h"

using namespace boost::python;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

不要引boost呀。。这个依赖太大了

@qingqing01
Copy link
Contributor Author

@reyoung cpplint及using namespace问题已经修改。 关于Boost问题, 一开始没有用, 后来看到Boost.Python用起来很方便,就改成Boost.Python了, 是否真的要去掉?如果真的要去掉的话,我就改下。

Copy link
Collaborator

@reyoung reyoung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

python文件的稳定性需要提升一些。

@@ -0,0 +1,170 @@
import os, psutil
import cv2
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

try:
   import cv2
except ImporError:
   log.warning("OpenCV2 is not installed, using single process image process instead")
   cv2 = None

如果cv2不存在,就退化到没有opencv一样的情况吧。

from paddle.utils.image_util import *
import multiprocessing
import subprocess, signal, sys

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

加上__all__字段,把需要Export的东西,Export出来。

channel_swap=None,
mean=None,
is_train=True,
is_color=True):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个MultiProcessImageTransfomer可以接一个参数,就是Transformer的类型是OpenCv的?还是Numpy的?

return self.transform(im)


class MultiProcessImageTransfomer():
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

继承自object

is_train=True,
is_color=True):
self.procnum = procnum
self.capacity = capacity
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

self.capacity 没有用

for i in xrange(self.procnum):
start = dlen * i / self.procnum
end = dlen * (i + 1) / self.procnum
proc = multiprocessing.Process(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里应该还可以再快些。。。因为args会多一步序列化,如果直接用shared_mem应该就会好一些。。不过也不好说。

except Exception as e:
print str(e)

def reset(self, size):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个文件如果不按照预期方式调用,可能会有一些隐患,例如process没停止等等。

比如,我一直调用run_proc,应该就挂了。。

def run(self, data, label):
try:
fun = partial(warpper, self)
return self.pool.imap_unordered(fun, zip(data, label), chunksize=5)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

有几个问题:

  • chunksize最好再设大一些,并且是procnum的倍数。例如 procnum * 100

    • 是procnum的倍数是因为这样各个进程间会比较均衡。
    • 再大一些是因为可以避免一些时候,某个核心被其他人占用,产生的抖动。
  • 推荐用itertools.izip替换zip。因为zip会先把list生成出来,然后再传递给imap_unordered函数,而izip会返回一个generator,一条一条的返回。这样首先内存占用应该理论上会小一些。同时,izip可以按照需要zip,应该会稍微快一些。

return self.pool.imap_unordered(fun, zip(data, label), chunksize=5)
except KeyboardInterrupt:
self.pool.terminate()
except Exception, e:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里不是阻塞的,所以这里try...except一点用应该都没有

else:
return self.transformer.transform_from_file(data), label

def __getstate__(self):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

不要加这个函数了

del self_dict['pool']
return self_dict

def __setstate__(self, state):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

不要加这个函数了

self.pool.terminate()
except Exception, e:
self.pool.terminate()

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@staticmethod
def __job__(is_img_string, transformer, data, label):
    if is_img_string:
       transformer. transform_from_string(data), label
    else:
       transformer. transform_from_file(data), label


def run(self, data, label):
try:
fun = partial(warpper, self)
Copy link
Collaborator

@reyoung reyoung Dec 5, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

func = partial(MultiProcessImageTransformer. __job__, self.is_img_string, self.transformer)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

另外,最好不要 from functools import partial 而是 import functoolsfunctools.partial
这样可读性强一点点。

return self.transform(im)


def warpper(cls, (dat, label)):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

删了吧,没用


from paddle.utils.image_util import *
from paddle.trainer.config_parser import logger

Copy link
Collaborator

@reyoung reyoung Dec 5, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

加入 __all__,管理export的符号。

@ghost
Copy link

ghost commented Dec 8, 2016

0@nva 1 no

@ghost
Copy link

ghost commented Dec 8, 2016

0@nva 1 no

Copy link
Collaborator

@reyoung reyoung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basically LGTM

start_w = (col - self.crop_size) / 2
end_h, end_w = start_h + self.crop_size, start_w + self.crop_size
im = im.crop((start_h, start_w, end_h, end_w))
if (self.is_train) and (np.random.randint(2) == 0):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里不需要加括号

return self.transform(im)

def load_image_from_file(self, file):
im = Image.open(file)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

最好不要使用file作为参数名,file在python里面是个函数

@reyoung reyoung merged commit 2039070 into PaddlePaddle:develop Dec 8, 2016
@ghost
Copy link

ghost commented Dec 8, 2016 via email

@qingqing01 qingqing01 deleted the acc_image_proc branch April 20, 2017 10:34
@Sampson1107
Copy link

@qingqing01
你好,是否有下面相关测试算例的文档或者demo?
PIL直接读取图片list: 64.8s
Pickle对数据打Batch再读取:46.8s
数据打Batch + Python多进程 + Python-OpenCV: 29.7s
数据打Batch + C++多线程 + OpenCV: 26.7s

zhhsplendid pushed a commit to zhhsplendid/Paddle that referenced this pull request Sep 25, 2019
* synchronize with develop (PaddlePaddle#642)

* update_commitid1.3 (PaddlePaddle#641)

* update inference c++ API doc (PaddlePaddle#634)

* update inference c++ API doc

* fix link

* thorough clean for doc (PaddlePaddle#644)

* thorough clean

* delete_DS_Store

* Cherrypick1.3 (PaddlePaddle#652)

* thorough clean

* delete_DS_Store

* [Don't merge now]update_install_doc (PaddlePaddle#643)

* update_install_doc

* follow_comments

* add maxdepth (PaddlePaddle#646)

* upload_md (PaddlePaddle#649)

* update_version (PaddlePaddle#650)

* Translation of 16 new apis (PaddlePaddle#651)

* fix_windows

* Final update 1.3 (PaddlePaddle#653)

* thorough clean

* delete_DS_Store

* update_1.3

* Deadlink fix (PaddlePaddle#654)

* fix_deadlinks

* update_docker

* Update release_note.rst

* Update index_cn.rst

* update_Paddle (PaddlePaddle#658)

* fix pic (PaddlePaddle#659)

* [to 1.3] cn api debug (PaddlePaddle#655) (PaddlePaddle#661)

* debug

* fix 2 -conv2d

* "锚" ==> anchor(s)

* Weekly cherrypick0302 (PaddlePaddle#668)

* Update programming_guide.md (PaddlePaddle#664)

* Update programming_guide.md

* Update programming_guide_en.md

* Update cn api to 1.3 (PaddlePaddle#663)

* Update cn api to 1.3 fluid & layers

* Rest to 1.3

* Weeklyupdate 0301 (PaddlePaddle#666)

* Tables_rm_op

* update_op

* update_index

* update_book_0302 (PaddlePaddle#667)

* fix_format (PaddlePaddle#669) (PaddlePaddle#670)

* fix_format

* Update Tables.md

* Update Tables_en.md

* add dataset api_cn (PaddlePaddle#673)

* rm fluid.core in desigin_idea (PaddlePaddle#674)

* Update fluid_design_idea.md

* Update fluid_design_idea_en.md

* Fix array_read code example error. (PaddlePaddle#671)

Signed-off-by: zhaoyuchen <[email protected]>

* add data_reader_cn (PaddlePaddle#676)

* fix doc error (PaddlePaddle#675)

* update_book_commitid (PaddlePaddle#680)

* update_book_commitid

* commitid0309

* fix typo

* book indexes (PaddlePaddle#677)
Meiyim pushed a commit to Meiyim/Paddle that referenced this pull request May 21, 2021
wangxicoding pushed a commit to wangxicoding/Paddle that referenced this pull request Dec 9, 2021
lizexu123 pushed a commit to lizexu123/Paddle that referenced this pull request Feb 23, 2024
WAYKEN-TSE pushed a commit to WAYKEN-TSE/Paddle that referenced this pull request Dec 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants